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INTRODUCTION 


In  February  of  1985,  The  BDM  Corporation  was  contracted  by  the  Army 
Research  Institute  (ARI)  to  conduct  a  three-year  research  effort  in  support  of 
ARI' s  mission  as  primary  research  agency  for  the  Army  at  the  National  Training 
Center  (NTC) .  Performance  of  this  contract  required  that  BDM  successfully 
complete  five  tasks  in  each  of  the  three  years  of  the  contract.  As  a  result  of  a 
contract  modification,  the  number  of  tasks  in  the  second  and  third  years  was 
increased  to  six  per  year.  Work  was  performed  on  site  at  ARI  Presidio  of  Monterey 
(ARI-POM)  . 

The  purpose  of  this  document  is: 

1.  to  provide  a  description  of  the  results  achieved  during  the 
contract;  and 

2.  to  provide  a  list  of  the  products  delivered  under  this 
contract . 

There  were  three  primary  thrusts  to  this  effort:  standardize  (to  enhance 
data  quality) ,  structure  (to  enhance  data  accessibility) ,  and  analyze  (to  enhance 
data  interpretation).  After  a  brief  overview,  the  discussion  of  BDM' s 
accomplishments  centers  around  the  three  outcome  areas:  data  quality,  data 
accessibility,  and  data  interpretation.  The  final  section  summarizes  the 
activities  and  lists  the  products  per  contract  task. 


OVERVIEW  OF  CONTRACT  EFFORT 

To  facilitate  an  understanding  of  the  products  and  activities  undertaken, 
it  is  important  to  understand  that  there  were  three  primary  thrusts  to  the 
contract  effort: 

1.  To  standardize  the  data  collection  procedures  at  the  NTC,  thus 
improving  the  quality  of  the  data  derived  from  the  NTC; 

2.  To  structure  the  data  from  the  NTC  to  make  it  more  accessible  to 
support  research  and  lessons  learned; 

3.  To  analyze  and  interpret  NTC  data  to  produce  products  and  reports,  as 
well  as  to  establish  methodologies  for  future  application  to  NTC 
data . 

The  contract  work  efforts  were  structured  and  performed  to  maximize  the 
benefit  to  be  accrued  by  both  sets  of  Army  proponents,  the  scientific  community 
and  the  military  community.  Each  step  of  the  research  process  was  used  to  generate 
results  that  satisfied  both  proponent  communities.  This  is  represented  in  Figure 
2.1. 
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Figure  2.1  The  research  process  systems  model 


The  basic  approach  taken  by  BDM  in  conducting  work  was  to  follow  the  flow 
of  logic  as  illustrated  in  Figure  2.2. 

In  1985,  many  of  the  procedures  and  operations  for  conducting  a  training 
rotation  at  the  NTC  were  undocumented,  highly  variable,  and  shifted  considerably 
with  changes  in  the  change  of  command.  Perhaps  even  more  importantly,  assessment 
of  unit  performance  was  unsystematic  and  largely  subjective,  based  on  anecdotal 
accounts  of  the  Observer/Controllers  (O/Cs) .  BDM  undertook  several  efforts  to 
contribute  to  greater  standardization  including  manuals  for  DeAnza  Workstation 
Operators  and  guidebooks  for  O/Cs .  However  the  largest  part  of  the  effort  to 
standardize  centered  on  the  development  of  a  system  for  measuring  and  viewing  unit 
performance  at  the  NTC.  This  system,  described  below,  has  come  to  be  known 
throughout  the  Army  as  "2A". 

In  1985,  digital  data  and  other  training  information  emerging  over  the 
course  of  NTC  rotations  were  largely  unusable  from  a  research  perspective.  The 
database  was  unwieldy  and  difficult  to  use,  and  contained  large  amounts  of  missing 
and  redundant  data.  As  part  of  the  contract  effort,  BDM  developed  a  fully 
operational,  fully  documented  database  containing  all  NTC  missions  since  1986. 

BDM  also  helped  ARI  develop  workshops  whereby  prospective  NTC  database  users  are 
trained  and  certified  to  access  the  information  in  the  database. 

With  the  development  of  a  usable  research  database,  ARI's  capability  to 
address  targeted  issues  was  expanded.  BDM  has  participated  in  numerous  analyses 
to  support  the  NTC  Lessons  Learned  research  program  administered  by  the  Center  for 
Army  Lessons  Learned  (CALL) ,  as  well  as  analyses  to  develop  concepts  and  methods 
such  as  trendline  analysis. 
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Analyze  unit  performance  and  then 
integrate  with 


Develop  guides  on  use  of 
NTC  performance  data 


•  Home  station  training  conditions 
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at  the  NTC 

•  Army-wide  needs  for  NTC  data  . 
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assessment 

•  Training  and  feedback  at  NTC 
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Figure  2.2  Approach  to  execution  of  contract 


A  more  detailed  presentation  of  accomplishments  and  products  is  now 
presented  for  data  quality,  data  accessibility,  and  data  interpretation. 


Data  Quality 

BDM  approached  the  data  quality  issue  with  the  objective  of  developing  a 
framework  to  serve  as  a  data  collection  and  analysis  instrument  that  would  offer  a 
meaningful  insight  into  systemic  issues  as  they  impact  on  the  tactical  employment 
of  maneuver  units . 

Task  2A  mandated  the  development  of  a  system  of  data  collection  that  could 
be  utilized  for  immediate  tactical  training  requirements  and  long  term  research. 
Specifically,  this  system  had  two  requirements.  The  first  requirement  was  that  the 
system  have  field  utilization  for  rapid  feedback  of  tactical  events  at  the  NTC. 
Secondly,  the  system  was  to  be  designated  for  use  by  research  and  military 
agencies  in  framing  and  analyzing  a  variety  of  tactical  systemic  issues  from 
training  to  synchronization. 

The  product  developed  by  BDM  to  address  these  requirements  is  officially 
entitled  the  Unit  Performance  Measurement  System,  commonly  referred  to  as  2A.  It 
was  developed  in  two  major  parts:  The  Unit  Effectiveness  Measurement  System  and 
the  Critical  Task  Measurement  System. 

The  Unit  Effectiveness  Measurement  System  (UEMS)  is  designed  to  give  an 
outcome  assessment  to  a  specific  mission.  Structured  against  the  factors  of  METT-T 
(Mission,  Enemy  forces,  Friendly  forces,  Terrain,  and  Time) ,  it  assigns  an 
attribute  of  condition,  purpose,  and  standard  to  each  factor  according  to  the 
respective  mission.  Although  the  UEMS  was  designed  specifically  for  the  battalion 
task  force  level,  its  structure  is  flexible  enough  for  use  at  all  echelons  up  to 
brigade  level. 

The  Critical  Task  Measurement  System  (CTMS)  is  designed  to  assess  a  unit 
conducting  a  tactical  mission  through  the  use  of  mission  critical  tasks.  These 
tasks,  which  are  grouped  within  an  echelon  and  mission  specific  framework,  are 
organized  both  sequentially  and  functionally  to  reflect  the  pattern  of  activity 
during  a  mission.  This  task  layout  facilitates  diagnosis  of  why  a  respective  unit 
did,  or  did  not,  do  well  on  a  particular  mission.  Mission  books  were  developed  for 
platoon,  company/team,  and  task  force  echelons  covering  five  mission  types: 

1.  Defend, 

2.  Deliberate  Attack  (Day), 

3.  Deliberate  Attack  (Night), 

4.  Movement  to  Contact,  and 

5 .  Hasty  Attack . 

Critical  tasks  were  defined  for  each  of  the  five  mission  types  by  echelon. 
Some  tasks  were  applicable  to  more  than  one  mission  type  while  others  were 
applicable  to  only  one.  Conditions  were  derived  from  an  in-depth  analysis  of  the 
METT-T  factors  using  the  NTC  environment  as  the  setting  for  this  study.  These 
conditions  were  provided  for  each  mission  type  and  its  phase  and  battle  segment. 
Standards  of  performance,  in  terms  of  measurable  outcomes,  were  also  provided  for 
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each  mission  type.  A  criterion  measurement  was  then  provided  for  each  critical 
task . 


In  summary,  2A  was  designed  to  serve  as  a  data  collection  instrument,  an 
immediate  mission  feedback  mechanism,  and  a  data  analysis  structure.  It  also  can 
facilitate  identification  of  tactical  training  requirements,  operational  trends, 
and  systemic  issues  requiring  resolution.  These  aspects  of  the  system  serve  to 
enhance  the  effectiveness  of  the  Combat  Training  Centers  and  tactical  training 
within  maneuver  battalions  throughout  the  Army. 


Data  Accessibility 

Much  of  the  effort  expended  during  the  previous  contract  was  aimed  at 
providing  ARI  and  the  Army  access  to  data  from  the  NTC.  The  three  year  effort 
provided  a  number  of  products  devoted  to  this  goal. 

The  initial  period  of  the  contract,  covering  approximately  18  months,  was 
dedicated  to  examination  of  the  NTC  range  data  collection  systems  and  the  existing 
research  database  system  vis  a  vis  the  research  requirements  of  ARI  and  the  Army. 

A  requirement  for  more  effective  user  documentation  for  NTC  and  research 
personnel,  who  were  called  upon  to  use  the  De  Anza  workstations,  was  identified 
and  satisfied.  A  concentrated  investigation  into  the  format  and  character  of  the 
NTC  data  was  conducted,  resulting  in  detailed  documentation  of  the  two  NTC  digital 
data  sources,  the  log  tapes  from  the  Range  Data  Measurement  Subsystem  (RDMS)  and 
files  from  the  Core  Instrumentation  Subsystem  (CIS) . 

The  next  major  step  in  enhancing  the  accessibility  of  NTC  data  was  the 
design  of  an  improved  research  database  system.  It  became  apparent  from  the 
activities  mentioned  above  that  a  redesign  was  necessary  for  several  reasons: 

1.  The  original  design  was  inefficient,  in  that  a  great  deal  of 
redundant  information  was  present  in  several  tables; 

2.  The  redundancy  caused  the  databases  to  be  inordinately  wasteful  of 
disk  space,  limiting  the  number  of  databases  that  could  be  maintained 
online  at  any  particular  time;  and 

3.  The  process  required  to  build  the  databases  was  unwieldy,  yet 
provided  little  or  no  opportunity  to  control  the  scope  of  the 
databases  that  were  built. 

The  database  development  effort  was  initiated  by  defining  an  approach  that 
would  culminate  in  a  database  system  that  supported  the  requirements  of  the  Army 
and  ARI.  The  next  step  was  the  publication  of  a  draft  'strawman'  preliminary 
design.  This  strawman  was  distributed  to  all  potential  users  of  the  NTC  data  which 
had  been  identified  at  that  time.  Comments  and  suggestions  were  considered  and 
incorporated  via  a  modified  Delphi  technique  into  the  final  revision  of  the  basic 
design,  and  development  commenced.  The  database  building  software  was  completed  in 
May,  1987,  at  which  time  BDM  began  to  build  mission  databases. 
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Subsequent  to  the  availability  of  the  redesigned  ARI-POM  Database 
Management  System,  a  need  was  identified  by  the  TRADOC  commander  to  make  use  of 
the  historical  NTC  data  to  generate  trendline  analyses  in  support  of  the  Center 
for  Army  Lessons  Learned  (CALL) .  Soon  afterward,  the  commanders  of  the  Army 
schools  were  urged  to  'get  smart'  about  the  NTC  data  available  for  analysis. 
Accordingly,  a  series  of  workshops  were  conducted  by  ARI-POM  to  prepare  delegates 
from  the  Army  school  system  to  make  use  of  the  NTC  data.  BDM  participated  in  the 
design  and  conduct  of  these  week-long  training  sessions.  In  addition  to  the 
courseware  developed  to  hold  the  series  of  workshops,  a  guide  was  developed  to 
using  the  ARI-POM  research  database. 


Data  Interpretation 

In  addition  to  enhancing  the  accessibility  and  quality  of  the  NTC  data,  BDM 
has  been  committed  to  the  large-scale  effort  involving  the  interpretation  of 
training  data.  This  commitment  to  ARI  programmatic  research  is  illustrated  by  a 
wide  array  of  BDM  research  reports  and  products  generated  during  the  contract 
period.  The  research  conducted  represent  three  similar,  yet  distinct,  types: 
methodological  research  intended  to  establish  appropriate  means  for  modeling  and 
analyzing  training  data;  research  reports  providing  analytical  support  to  CALL  on 
high-priority  training  issues;  and,  highly-specialized  applied  research  products 
that  are  adopted  for  use  within  the  Army  training  environment. 

The  purpose  of  methodological  research  is  to  develop  concepts  and  methods 
for  measurement,  interpretation,  and  feedback  of  CTC  data.  Some  of  the  specific 
objectives  accomplished  during  the  contract  were  to  develop  concepts  and  methods 
for  measurement  and  interpretation  of  unit  performance  at  the  NTC,  develop 
concepts  for  trendline  analysis,  and  develop  an  integrated  framework  for  CTC 
research . 

The  purpose  of  the  NTC  Lessons  Learned  research  program  is  to  provide 
analytical  support  to  CALL  high-priority  issues.  The  Lessons  Learned  research 
efforts  were  directed  by  CALL  as  a  result  of  heightened  interest  in  particular 
phenomena  occurring  during  NTC  training.  Research  findings  often  resulted  in  the 
designation  of  focused  rotations  at  the  NTC  or  further  study  of  related  issues. 
During  the  course  of  the  three-year  contract,  several  Lessons  Learned  reports  were 
produced.  Lessons  Learned  analyses  included  the  use  of  instrumented  data,  Take 
Home  Packages  (THPs) ,  After  Action  Review  (AAR)  video  tapes,  doctrinal  manuals, 
and  data  displays  developed  by  BDM  at  ARI-POM.  Lessons  Learned  findings  were 
presented  as  reports,  briefings,  or  as  contributions  to  larger-scale  efforts  being 
conducted  by  CALL. 

The  third  category  of  research  conducted  by  ARI/BDM  involves  the  production 
of  applied  research  products  that  are  used  for  Army  training  purposes.  Observer/ 
Controller  Guidebooks,  developed  jointly  by  ARI  and  BDM  are  currently  being  used 
by  the  observer/  controllers  at  the  NTC.  A  training  video  tape  was  developed  to 
demonstrate  how  NTC  data  can  be  optimally  used  for  training  combat  leaders.  The 
video  is  composed  of  training  vignettes  which  are  composed  of  De  Anza  workstation 
battle  graphics,  portions  of  communications  tapes,  and  AAR  segments  that  are 
designed  to  illustrate  critical  points  occurring  in  actual  NTC  battle  segments. 
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In  summary,  BDM  has  supported  efforts  encompassing  the  many  aspects  of 
ARI's  programmatic  training  research. 


CONTRACT  TASK  SUMMARIES 

The  final  section  of  this  report  provides  a  brief  summary  of  activities  and 
a  product  list  per  individual  contract  task.  Also  presented  for  each  task  is  a 
description  of  how  the  contract  products  have  been  utilized.  The  contract  tasks 
are  grouped  by  year  of  the  contract  (1  through  3) .  The  structure  of  the  contract 
was  such  that  the  output  from  each  year' s  effort  was  designed  to  feed  directly 
into  the  conduct  of  the  next  year's  tasks.  Tasks  1  through  5  were  accomplished 
during  Year  One;  Tasks  6  through  10,  during  Year  Two;  and  Tasks  11  through  15 
during  Year  3.  Task  2A,  a  task  added  through  a  contract  modification,  was 
designed  to  be  accomplished  during  years  2  and  3.  Year  Three  was  18  months  long, 
due  to  an  extension  through  August  1988. 

The  deliverables  are  presented  under  the  contract  year  heading  in  which  the 
work  was  performed  and  the  product  delivered;  in  some  cases  the  ARI  publication 
date  for  a  deliverable  may  correspond  to  a  later  contract  year.  No  report  number 
is  listed  for  documents  which  are  not  yet  final  ARI  publications. 
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Contract  Year  One 
(Tasks  1  through  5) 


Task  1 


Develop  and  Apply  Procedures  for  Identification 
and  Correction  of  Erroneous  Data  from  NTC 


Summary 

This  task  had  as  its  central  concern  the  investigation  of  the  quality  of 
the  digital  data  from  the  NTC.  As  part  of  this  effort,  several  comparisons  were 
undertaken  to  determine  the  relative  accuracy  of  varying  data  sources. 


Product s /Report s 

Briscoe,  J.A.  (1987,  January) .  Capability  to  analyze  National  Training 

Center  data  (ARI  Research  Note  87-08).  Alexandria,  VA:  U.S.  Army 
Research  Institute.  [DTIC  #  ADA178948] 

Briscoe,  J.A.  (1987,  January) .  Comparison  of  National  Training  Center  data 
sources  (ARI  Research  Note  87-12).  Alexandria,  VA:  U.S.  Army 
Research  Institute.  [DTIC  #  ADA179797] 

Briscoe,  J.A.  (1987,  January) .  Program  GDETAP  documentation  (ARI  Research 

Note  87-09) .  Alexandria,  VA:  U.S.  Army  Research  Institute.  [DTIC  # 
ADA17  98  92 ] 

Kemper,  T.  R.,  &  Briscoe,  J.A.  (1987,  January).  National  Training  Center 

Data  Library  (ARI  Research  Note  87-07).  Alexandria,  VA:  U.  S.  Army 
Research  Institute.  [DTIC  #  ADA178947] 


Utilization 


The  products  produced  under  this  task  have  been  used  to  guide  research 
efforts  and  to  document  computer  code.  They  have  also  provided  the  information 
needed  for  informed  decision-making  regarding  expansion  and  improvement  of  the 
NTC  data  archive  at  ARI-POM.  The  document  entitled  "Capability  to  Analyze 
National  Training  Center  Data"  was  used  as  a  source  document  by  researchers  to 
guide  their  research  efforts.  It  was  also  used  as  a  jumping-off  point  to  design 
improvements  in  the  ARI -NTC  data  base  design  effort.  The  document  entitled 
"Comparison  of  National  Training  Center  Data  Sources"  defined  the  similarities  and 
differences  between  the  two  major  instrumented  NTC  data  archives.  It  served  as  the 
basis  for  the  decision  to  begin  archiving  RDMS  tapes  at  ARI-POM,  and  to  include 
RDMS  data  in  the  mission  databases.  "Program  GDETAP  Documentation"  documents  the 
code  used  to  read  the  NTC  RDMS  tapes  and  to  produce  the  three  standard  reports. 

The  document  entitled  "National  Training  Center  Data  Library"  served  as  a  basis 
to  expand  and  improve  the  storage  of  NTC  data  at  ARI-POM. 
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Task  2 


Develop  and  Apply  Concepts  and  Methods  for  Measurement 
and  Interpretation  of  Unit  Performance 


Summary 

Defining  and  measuring  battalion  combat  effectiveness  was  the  central  issue 
for  this  task.  Preliminary  analyses  of  live-fire  performance,  force-on-force 
performance,  and  relative  performance  of  BFVs  and  M113s  were  prepared  to 
demonstrate  the  utility  of  NTC  data  for  describing  combat  performance.  At  the 
same  time  a  concept  and  methodology  were  developed  for  a  systems  approach  to 
measuring  combat  performance.  A  contract  modification  was  prepared  to  permit 
development  of  the  proposed  system  for  all  NTC  missions  rather  than  the  prototype 
mission  originally  called  for  in  the  contract.  This  task  is  described  under 
contract  years  2  and  3  and  is  entitled  Task  2A. 


Products/Reports 

Forsythe,  T.  K.  (1987,  January) .  A  research  concept  for  developing  and 
applying  methods  for  measurement  and  interpretation  of  unit 
performance  at  the  NTC  (ARI  Research  Report  1435) .  Alexandria,  VA: 
U.S.  Army  Research  Institute. 

Ritenour,  T.  J.  (1987,  January) .  National  Training  Center  research  issues 
(ARI  Research  Note  87-06)  Alexandria,  VA:  U.S.  Army  Research 
Institute . 

Doherty,  W.  J.,  &  Hiller,  J.  H.  (1986,  January).  Comparison  of  the  BFVs  and 
M113  equipped  battalion  task  forces  on  live  fire  performance  at  the 
NTC  (ARI  Special  Report)  .  Presidio  of  Monterey,  CA:  U.S.  Army 
Research  Institute  Field  Unit. 

Forsythe,  T.  K.,  &  Doherty,  W.J.  (1987,  April).  An  NTC  live  fire 

performance  analysis  (ARI  Research  Note  87-21) .  Alexandria,  VA: 

U.S.  Army  Research  Institute.  [DTIC  #  ADA188247] 

Nichols,  J.  J.  (1987,  February) .  A  preliminary  analysis  of  NTC  force-on- 
force  performance  (ARI  Research  Note  87-14) .  Alexandria,  VA:  U.S. 
Army  Research  Institute.  [DTIC  #  ADA180287] 


Utilization 


Concepts  developed  in  these  efforts  were  used  to  generate  the  initial 
empirical  results  from  the  NTC.  Specifically,  the  reports  on  live  fire 
performance  and  force-on-force  performance  were  the  first  empirically  based 
results  from  the  NTC  provided  to  CALL.  The  report  on  the  BFVs  provided  input  to 
the  Army's  response  to  congressional  queries  concerning  BFV  effectiveness.  The 
Forsythe  report  provided  the  initial  systems  concept  that  led  to  the  work 
performed  under  Task  2A  of  this  contract. 
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Task  3 


Develop  Guides  to  the  Use  of  NTC  Information 


Summary 

Work  on  this  task  followed  two  different  research  strands.  One  strand  led 
to  the  creation  and  production  of  a  prototype  document  entitled  "Lessons  from 
NTC."  The  other  strand  was  concerned  with  the  preparation  of  guides  for  use  with 
the  NTC  instrumentation  system. 


Products /Report s 


Ritenour,  T.  J.  (1987,  January)  A  detailed  description  of  the  National 

Training  Center  instrumentation  system  initialization  procedure  (ARI 
Research  Note  87-03)  .  Alexandria,  VA:  U.S.  Army  Research  Institute. 
[DTIC  #  ADA1 79860] 

Nichols,  J.  J.  (1987,  January) .  The  De  Anza  primer:  a  basic  introduction  to 
the  De  Anza  graphics  display  (ARI  Research  Note  87-01) .  Alexandria, 
VA:  U.S.  Army  Research  Institute.  [DTIC  #  ADA179798] 

Avant,  T.  L.,  &  Henderson,  R.  S.  (1986,  July)  Documentation  for  'What  Now, 
Captain?' :  a  training  concept  for  exporting  Lessons  Learned  from  the 
National  Training  Center  (ARI  Research  Note  87-13) .  Alexandria,  VA: 
U.S.  Army  Research  Institute.  [DTIC  #  ADA179991] 

Shackelford,  W.  L.  (1987,  January) .  A  Method  of  analysis  for  the  Bradley 
Fighting  Vehicle  System  (ARI  Research  Note  87-10) .  Alexandria,  VA: 
U.S.  Army  Research  Institute. 

Shackelford,  W.  L.  (1985,  November) .  Lessons  from  the  NTC  (ARI  Research 
Report).  Presidio  of  Monterey,  CA:  U.S.  Army  Research  Institute 
Field  Unit. 


Utilization 


The  work  performed  on  the  NTC  instrumentation  system  and  the  DeAnza 
workstation  were  both  aimed  at  assisting  the  NTC  Operations  Group  in  its  internal 
training  program.  Further,  both  provided  documentation  that  were  incorporated 
into  system  documentation  at  the  NTC.  The  remaining  work  provided  specific 
examples  of  applications  of  NTC  data  for  high  priority  Army  issues.  The  "What 
Now,  Captain?"  concept  provided  the  first  example  of  how  to  bring  the  lessons  from 
the  NTC  into  the  classroom.  The  "Lessons  from  the  NTC"  provided  the  first 
comprehensive  view  of  military  lessons  from  a  tactical  expert's  perspective. 
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Task  4 


Determine  Requirements  for  More  Effective  Integration 
of  NTC  and  Home  Station  Training 


Summary 

By  capitalizing  on  the  measurement  system  developed  in  Task  2  and  the 
result  from  "Lessons  from  NTC, "  the  effort  undertaken  in  this  task  provided  input 
into  Home  Station  Training  factors  related  to  NTC  performance. 


Products /Reports 

Kerins,  J.  W.,  Forsythe,  T.  K.,  &  Avant,  T.  L.  (1987,  January). 

Research  plan  for  the  evaluation  of  the  requirements  for  the 
effective  integration  of  the  National  Training  Center  and  Home 
Station  Training  (ARI  Research  Note  87-11).  Alexandria,  VA:  U.S. 
Army  Research  Institute.  [DTIC  #  ADA179893] 


Utilization 


This  report  provided  a  blueprint  for  an  approach  to  researching  the 
systematic  link  between  home  station  and  NTC  performance.  Much  of  this  approach 
has  been  included  in  the  ongoing  ARI  effort  directed  at  Determinants  of  Combat 
Effectiveness . 
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Task  5 


Prepare  Proposed  Research  Plan  for  Year  2 


Summary 

As  per  contract  requirement,  a  research  plan  was  prepared  which 
incorporated  Year  One  results  along  with  the  originally  proposed  plans  for  conduct 
of  Year  Two.  This  plan  was  originally  entitled  Proposed  Research  Plan  for  Year 
Two  of  the  Contract. 


Products /Reports 

Doherty,  W.  J.  (Ed.)  (1987,  February).  Methodology  development  for  deriving 

lessons  learned  from  the  National  Training  Center;  Progress  and 
future  directions  (ARI  Research  Note  87-16) .  Alexandria,  VA:  U.  S. 
Army  Research  Institute.  [DTIC  #  ADA180026] 
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Contract  Year  Two 
(Tasks  2A,  6  through  10) 


Task  2A  (CY  Two) 

Develop  Concepts  and  Methods  for  the  Measurement 
and  Interpretation  of  Unit  Performance 


Summary 

This  task  represented  an  expansion  and  continuation  of  Task  2  above.  It 
involved  two  central  activities:  the  creation  of  a  criterion  measurement  system 
for  determining  combat  effectiveness  at  the  NTC  and  the  identification  and 
measurement  of  critical  tasks  at  the  NTC. 


Products /Reports 


Lewman,  T.  J.  (1987,  May) .  Mission  critical  tasks  at  the  National  Training 
Center  (ARI  Research  Product) .  Presidio  of  Monterey,  CA:  U.S.  Army 
Research  Institute  Field  Unit. 

Root,  J.  T.,  &  Zimmerman,  R.  A.  (1988,  August).  A  unit  effectiveness 

measurement  system  at  the  National  Training  Center  (ARI  Research 
Product).  Presidio  of  Monterey,  CA:  U.S.  Army  Research  Institute 
Field  Unit. 


Utilization 


Critical  task  lists  for  the  task  force,  company/team,  and  platoon  have  been 
integrated  into  Infantry  and  Armor  Mission  Training  Plans.  The  mission  critical 
tasks  were  used  as  the  basis  for  further  development  of  the  Critical  Task 
Measurement  System  (CTMS) .  The  Unit  Effectiveness  Measurement  System  (UEMS) 
provides  the  means  to  assess  task  force  mission  performance  for  those  units 
training  at  the  NTC.  These  data  are  used  by  the  Army  for  analyzing  training 
performance  trends. 
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Task  6 


( 

Analyze  Unit  Performance 


Summary 

This  task  had  as  its  major  focus  the  design  and  creation  of  a  research 
database  for  NTC  data  at  the  ARI  Presidio  of  Monterey  (POM)  facility.  Guides  to 
the  use  of  the  database  were  developed  as  part  of  this  effort. 

Product s /Reports 


Briscoe,  J.  A.  (1987,  December) .  ARI-POM  database  development  plan  (ARI 

Research  Note  87-74.  Alexandria,  VA:  U.S.  Army  Research  Institute. 
[DTIC  #  ADA190586] 

Briscoe,  J.  A.,  &  Baldwin,  J.  D.  (1987,  April).  NTC  tactical  database 

preliminary  design  (ARI  Research  Note  87-75) .  Alexandria,  VA:  U.S. 
Army  Research  Institute.  [DTIC  #  ADA190585] 

Briscoe,  J.  A.  &  Baldwin,  J.  D.  (1987,  March)  Programmer' s  guide  to  data 
from  the  National  Training  Center  (ARI  Research  Product) .  Presidio 
of  Monterey,  CA:  U.S.  Army  Research  Institute  Field  Unit. 


Utilization 

The"ARI-POM  Database  Development  Plan”  laid  out  the  steps  in  the  development  of 
the  NTC  mission  database  system,  and  was  used  as  a  basis  for  that  development 
effort.  The  document  entitled  "NTC  Tactical  Database  Preliminary  Design”  laid  out 
the  proposed  table  formats  for  the  NTC  mission  databases  and  was  later  upgraded  to 
reflect  the  implemented  versions.  It  has  been  used  extensively  by  personnel  who 
have  used  the  research  database  system  as  a  road  map  to  the  system.  It  has  been 
included  as  part  of  the  NTC  data  workshops  since  their  inception.  The 
"Programmer's  Guide  to  data  from  the  National  Training  Center"  has  been  widely 
used  by  researchers  and  others  interested  in  NTC  data  for  understanding  which  data 
elements  are  collected  by  the  NTC  instrumentation  system,  how  often  they  are 
collected,  and  how  they  are  organized. 
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Task  7 


Develop  Products  to  Support  Training 
and  Feedback  at  the  NTC 


Summary 

The  primary  focus  of  this  task  was  the  design  and  development  of  guidelines 
for  the  After  Action  Review  (AAR)  tapes  and  the  Take  Home  Packages  (THP)  at  the 
NTC.  To  accomplish  this,  the  effort  addressed  the  exploitation  of  the  advanced 
automation  capabilities  of  the  NTC  and  ARI-POM  such  as  the  content  and  format  of 
these  feedback  devices. 


Products /Reports 

Nichols,  J.  J.  (1987,  March).  NTC  Take  Home  Package:  1982-1986  (ARI 

Research  Report) .  Presidio  of  Monterey,  CA:  U.S.  Army  Research 
Institute  Field  Unit. 

Rachford,  D.  L.,  Twohig,  P.  T.,  &  Zimmerman,  R.  A.  (1986,  December).  The 
platoon  leader  -  platoon  sergeant  relationship  in  a  tactical 
environment:  focused  rotation  at  the  National  Training  Center  (NTC) 
(ARI  IiMTA  Working  Paper  86-06)  .  [Leadership  &  Management  Technical 
Area]  Alexandria,  VA:  U.S.  Army  Research  Institute. 

Zimmerman,  D.  C.,  &  Kemper,  T.  R.  (1987,  April).  The  NTC  After  Action 

Review:  implications  for  effective  training  feedback  (ARI  Research 
Report) .  Presidio  of  Monterey,  CA:  U.S.  Army  Research  Institute 
Field  Unit. 


Utilization 


The  examination  of  NTC  After  Action  Reviews  and  Take  Home  Packages  served 
to  provide  CATA  and  NTC  officials  with  information  about  the  quality  and  format  of 
these  feedback  tools.  Implications  from  these  studies  is  available  to  training 
coordinators  to  improve  the  manner  in  which  these  feedback  mechanisms  are  prepared 
and  presented  so  as  to  maximize  their  effectiveness  in  the  overall  training  and 
learning  experiences  of  soldiers  training  at  the  NTC. 

The  study  of  platoon  leader  and  platoon  sergeant  interaction  was  conducted 
in  support  of  an  NTC  focused  rotation  on  NCO  leadership.  The  findings  from  this 
study  were  used  by  CATA  to  produce  Lessons  Learned  material  that  appeared  in  a 
CALL  Lessons  Learned  publication  highlighting  platoon  leader  and  sergeant 
interaction.  The  study's  finding  were  also  used  as  a  basis  for  further  ARI 
research  on  leadership  at  the  NTC. 
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Task  8 


Develop  Products  to  Support  Home  Station  Training  and 
Training  Assessment 


Summary 

This  task  involved  the  development  of  data  collection  instruments  and 
techniques  for  use  in  describing  Home  Station  Training.  The  instruments  and 
techniques  drew  upon  the  measurement  system  being  developed  in  Task  2A  and  upon  a 
schema  of  the  Army  Training  Process  created  as  part  of  this  task. 


Product s /Reports 


Forsythe,  T.  K.,  &  Avant,  T.  L.  (1987,  July).  Methodology  development  for 
investigating  effective  NTC  and  home  station  training  integration 
(ARI  Working  Paper).  Presidio  of  Monterey,  CA:  U.S.  Army  Research 
Institute  Field  Unit. 


Utilization 


This  effort  operationalized  much  of  the  effort  outlined  in  Task  4  and 
produced  specific  data  collection  approaches,  instruments,  and  schedules.  All  of 
these  were  linked  through  the  2A  measurement  system  and  have  influenced  the 
important  ARI  effort  directed  at  Determinants  of  Combat  Effectiveness. 
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Task  9 


Develop  Products  to  Support  Feedback  to  Army  Users  and 
Army-Wide  Utilization  of  the  Products  of  the  Research 


Summary 

This  task  was  concerned  with  dissemination  of  NTC  information  to  the  Army 
at  large.  Several  Lessons  Learned  issues  selected  by  CATA  were  investigated. 


Product s/Reports 

Doherty,  W.  J.,  &  Atwood,  N.  K.  (1987,  February).  Killer  tank  crews  at  the 
National  Training  Center  (ARI  Working  Paper) .  Presidio  of  Monterey, 
CA:  U.S.  Army  Research  Institute  Field  Unit. 

Ritenour,  T.  J.  (1986,  March) .  A  report  of  battalion  task  force 

demonstrated  strengths  and  weaknesses  during  conduct  of  NTC  simulated 
combat  training  (ARI  Research  Report) .  Presidio  of  Monterey,  CA: 

U.S.  Army  Research  Institute  Field  Unit. 

Doherty,  W.  J.  &  Atwood,  N.  K.  (1987,  March) .  Commander  survivability  at 

the  National  Training  Center  (ARI  Special  Report  88-06) .  Alexandria, 
VA:  U.S.  Army  Research  Institute. 


Utilization 


Results  generated  under  this  task  received  wide  distribution  throughout 
CATA  and,  in  the  case  of  "Commander  Survivability",  throughout  the  Army.  The 
results  from  the  latter  were  distributed  to  all  Division  commanders  for  their 
consideration.  The  "Killer  Tank"  results  were  provided  to  both  the  CATA  and  CAC 
commanders.  In  all  three  cases,  the  results  were  incorporated  by  CALL  into  their 
dissemination  system  for  Army-wide  lessons  learned. 
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Task  10 


Prepare  Proposed  Research  Plan  for  Year  3 


Summary 


As  per  contract  requirement,  a  research  plan  was  prepared  which 
incorporated  Year  Two  results  along  with  the  originally  proposed  plans  for  conduct 
of  Year  Three. 


Products /Reports 

Doherty,  W.  J.  (Ed.)  (1987,  June).  Summary  of  Year  Two  activities  and 

research  plan  for  Year  Three  of  the  contract  (ARI  Research  Note) . 
Presidio  of  Monterey,  CA:  U.S.  Army  Research  Institute  Field  Unit. 
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CONTRACT  YEAR  THREE 
(Tasks  2A,  11  through  15) 


Task  2A  (CY  Three) 

Develop  Concepts  and  Methods  for  the  Measurement 
and  Interpretation  of  Unit  Performance 


Summary 

The  work  performed  on  Task  2A  in  the  final  year  of  the  contract  was  a 
continuation  of  the  effort  started  in  Task  2  Year  One  and  Task  2A  Year  Two.  The 
critical  task  measurement  system  was  developed  for  fifteen  echelon-mission 
combinations,  and  the  criterion  measurement  system  was  further  exercised. 


Products/Reports 

Lewman,  T.  J.,  &  Root,  J.  T.,  Zimmerman,  R.  A.  (1988,  August).  Critical 

Task  Performance  Measurement  System  (ARI  Research  Report) .  Presidio 
of  Monterey,  CA:  U.S.  Army  Research  Institute  Field  Unit. 

Lewman,  T.  J.,  &  Root,  J.  T.  (1987,  December).  Critical  Task  Performance 
Measurement  System  Mission  Book  -  Task  Force  Defend  (ARI  Research 
Note) .  Presidio  of  Monterey,  CA:  U.S.  Army  Research  Institute  Field 
Unit . 


Lewman,  T.  J.,  &  Root,  J.  T.  (1987,  December).  Critical  Task  Performance 
Measurement  System  Mission  Book  -  Task  Force  Deliberate  Attack  Day 
(ARI  Research  Note) .  Presidio  of  Monterey,  CA:  U.S.  Army  Research 
Institute  Field  Unit. 

Lewman,  T.  J.,  &  Root,  J.  T.  (1987,  December).  Critical  Task  Performance 

Measurement  System  Mission  Book  -  Task  Force  Deliberate  Attack  Night 
(ARI  Research  Note).  Presidio  of  Monterey,  CA:  U.S.  Army  Research 
Institute  Field  Unit. 

Lewman,  T.  J.,  &  Root,  J.  T.  (1987,  December).  Critical  Task  Performance 

Measurement  System  Mission  Book  -  Task  Force  Movement  to  Contact  (ARI 
Research  Note).  Presidio  of  Monterey,  CA:  U.S.  Army  Research 
Institute  Field  Unit. 

Lewman,  T.  J.,  &  Root,  J.  T.  (1987,  December).  Critical  Task  Performance 
Measurement  System  Mission  Book  -  Task  Force  Hasty  Attack  (ARI 
Research  Note).  Presidio  of  Monterey,  CA:  U.S.  Army  Research 
Institute  Field  Unit. 
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Lewman,  T.  J.,  &  Root,  J.  T.  (1987,  December).  Critical  Task  Performance 
Measurement  System  Mission  Book  -  Company  Team  Defend  (ARI  Research 
Note) .  Presidio  of  Monterey,  CA:  U.S.  Army  Research  Institute  Field 
Unit. 

Lewman,  T.  J.,  &  Root,  J.  T.  (1987,  December).  Critical  Task  Performance 

Measurement  System  Mission  Book  -  Company  Team  Deliberate  Attack  Day 
(ARI  Research  Note).  Presidio  of  Monterey,  CA:  U.S.  Army  Research 
Institute  Field  Unit. 

Lewman,  T.  J.,  &  Root,  J.  T.  (1987,  December).  Critical  Task  Performance 
Measurement  System  Mission  Book  -  Company  Team  Deliberate  Attack 
Night  (ARI  Research  Note) .  Presidio  of  Monterey,  CA:  U.S.  Army 
Research  Institute  Field  Unit. 

Lewman,  T.  J.,  &  Root,  J.  T.  (1987,  December).  Critical  Task  Performance 
Measurement  System  Mission  Book  -  Company  Team  Movement  to  Contact 
(ARI  Research  Note).  Presidio  of  Monterey,  CA:  U.S.  Army  Research 
Institute  Field  Unit. 

Lewman,  T.  J.,  &  Root,  J.  T.  (1987,  December).  Critical  Task  Performance 
Measurement  System  Mission  Book  -  Company  Team  Hasty  Attack  (ARI 
Research  Note).  Presidio  of  Monterey,  CA:  U.S.  Army  Research 
Institute  Field  Unit. 

Lewman,  T.  J.,  &  Root,  J.  T.  (1987,  December).  Critical  Task  Performance 

Measurement  System  Mission  Book  -  Platoon  Defend  (ARI  Research  Note) . 
Presidio  of  Monterey,  CA:  U.S.  Army  Research  Institute  Field  Unit. 

Lewman,  T.  J.,  &  Root,  J.  T.  (1987,  December).  Critical  Task  Performance 

Measurement  System  Mission  Book  -  Platoon  Deliberate  Attack  Day  (ARI 
Research  Note).  Presidio  of  Monterey,  CA:  U.S.  Army  Research 
Institute  Field  Unit. 

Lewman,  T.  J.,  &  Root,  J.  T.  (1987,  December).  Critical  Task  Performance 
Measurement  System  Mission  Book  -  Platoon  Deliberate  Attack  Night 
(ARI  Research  Note).  Presidio  of  Monterey,  CA:  U.S.  Army  Research 
Institute  Field  Unit. 

Lewman,  T.  J.,  &  Root,  J.  T.  (1987,  December).  Critical  Task  Performance 
Measurement  System  Mission  Book  -  Platoon  Movement  to  Contact  (ARI 
Research  Note).  Presidio  of  Monterey,  CA:  U.S.  Army  Research 
Institute  Field  Unit. 

Lewman,  T.  J.,  &  Root,  J.  T.  (1987,  December).  Critical  Task  Performance 

Measurement  System  Mission  Book  -  Platoon  Hasty  Attack  (ARI  Research 
Note) .  Presidio  of  Monterey,  CA:  U.S.  Army  Research  Institute  Field 
Unit . 


Root,  J.  T.  (November,  1987) .  Case  Study:  Functional  application  of  2A  (ARI 
Working  Paper).  Presidio  of  Monterey,  CA:  U.S.  Army  Research 
Institute  Field  Unit. 
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Root,  J.  T.  (December,  1987)  .  Extract  of  Data  Sources:  Armor  Task  Force, 
Deliberate  Attack  (ARI  Working  Paper) .  Presidio  of  Monterey,  CA: 
U.S.  Army  Research  Institute  Field  Unit. 


Utilization 


Task  2A' s  methodology,  models,  and  procedures  (sometimes  referred  to  as  the 
"2A  measurement  system")  are  being  incorporated  into  the  Army's  "how  to"  training 
circulars.  Training  developers  use  its  paradigms  as  a  guide  in  designing  Mission 
Training  Plans.  Task  2A' s  products  will  be  integrated  into  the  National  Training 
Center's  measurement  system.  The  methodology  is  being  used  to  guide  development 
of  a  similar  type  system  for  the  Combat  Maneuver  Training  Center  in  Germany.  A 
modified  version  of  2A  is  also  being  developed  for  use  with  the  interactive 
simulator  networking  system  (SIMNET) .  The  Combat  Training  Centers'  data,  stored 
in  ARI-POM  archives,  will  eventually  be  formatted  to  reflect  the  2A  structure. 
This  will  readily  simplify  data  searches  concerning  task  accomplishment  in 
relation  to  a  specific  echelon,  operating  system,  battle  phase  and  mission. 
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Task  11 


Develop  an  NTC  Research  Database 


Summary 

This  task  had  as  its  major  focus  the  loading  of  the  Mission  Database  and 
the  design  and  implementation  of  the  Technical  Database.  User  and  operator  guides 
were  developed  for  both  databases. 


Products /Reports 

Briscoe,  J.  A.  (1988,  August) .  Operators'  guide  to  loading  the  NTC  mission 
database  (ARI  Research  Product).  Presidio  of  Monterey,  CA:  U.S. 
Army  Research  Institute  Field  Unit. 

Baldwin,  J.  D.  (1988,  July).  Guide  to  using  the  ARI -NTC  research  database 
(ARI  Research  Product).  Presidio  of  Monterey,  CA:  U.S.  Army 
Research  Institute  Field  Unit. 


Utilization 


The  Operators'  Guide  has  been  used  to  introduce  operators  to  the  process  of 
building  mission  databases.  The  Researchers'  guide  is  included  in  the  NTC 
database  workshops  to  introduce  researchers  to  the  NTC  mission  databases  and  to 
provide  basic  methods  for  accessing  and  using  the  data. 
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Development  of  Concepts  for  Trendline  Analyses 


Summary 

This  task  continued  the  work  begun  under  Task  7  in  examining  methods  for 
improving  diagnostic  feedback  from  the  NTC.  Work  involved  the  development  of 
baselines  regarding  direct  fire  weapons  and  relationships  of  firing  intensity  to 
number  of  kills. 


Products /Reports 

Zimmerman,  R.  A.  (1987,  September) .  Trendline  analysis  of  direct  fire  at 
the  National  Training  Center  (ARI  Research  Report) .  Presidio  of 
Monterey,  CA:  U.S.  Army  Research  Institute  Field  Unit. 


Utilization 


BDM' s  development  of  trendline  analysis  methodologies  at  ARI-POM  was  used, 
in  part,  to  produce  Lessons  Learned  material  that  was  disseminated  to  soldiers 
training  at  the  NTC  through  CATA' s  Lessons  Learned  publications.  Some  of  the 
trendline  analyses  of  direct  fire  at  the  NTC  were  incorporated  into  ARI  briefings 
and  have  formed  the  basis  for  further  Army  trendline  studies.  In  addition, 
summary  slides  on  trendline  analysis  of  direct  fire  were  prepared  to  brief  GEN 
Thurman  at  the  Army  Research  Board  meeting  in  March  1988. 


Task  13 


Development  of  an  Integrated  Framework  for  CTC  Research 


Summary 

The  anticipated  information  flow  involving  NTC,  JRTC,  CMTC  and  BCTP  data 
was  mapped  into  a  strawman  taxonomic  structure  to  be  used  for  generating  issues, 
data  organization,  and  archiving  the  results.  A  briefing  was  prepared  and 
submitted  to  ARI . 


Products /Reports 

Zimmerman,  D.  C.  (1988,  August) .  Development  of  an  integrated  framework  for 
Combat  Training  Center  (CTC)  programmatic  research  (ARI  Research 
Product).  Presidio  of  Monterey,  CA:  U.S.  Army  Research  Institute 
Field  Unit. 


Utilization 


The  intended  use  of  this  research  product  is  to  provide  a  framework  for 
integrating  CTC  research.  This  framework  for  programmatic  research  integration 
was  presented  to  ARI  for  adoption.  The  framework  incorporates  the  automated  data 
catalog  currently  under  development  at  ARI-POM. 
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Task  14 


Support  of  NTC  Lessons  Learned 


Summary 

Analytic  support  for  the  CALL  Lessons  Learned  program  produced  several 
major  reports  in  response  to  top  priority  issues  as  directed  by  CALL.  The 
principal  effort  was  concerned  with  Commander  Survivability  at  the  NTC.  A 
methodology  for  showing  battle  intensity  profiles  was  developed  in  support  of  this 
effort . 


Products /Reports 

Zimmerman,  D.  C.  (1988,  May) .  A  study  of  commander  survivability  at  the 

National  Training  Center  (NTC)  (ARI  Research  Product) .  Presidio  of 
Monterey,  CA:  U.S.  Army  Research  Institute  Field  Unit. 


Utilization 


This  technical  report  provides  brief  background  information  on  the 
importance  of  commander  survival  and  summarizes  the  analyses  and  findings  from  the 
study  of  commander  survivability  at  the  NTC.  Findings  were  published  in  CATA' s 
Lessons  Learned  (88-1)  that  featured  NTC  commander  survivability  issues.  Summary 
slides  comparing  and  contrasting  doctrine  and  research  findings  were  prepared  for 
the  Army  Research  Board  briefing  of  GEN  Thurman  in  March  1988.  In  addition,  the 
battle  intensity  profiles  developed  for  this  task  were  used  in  NTC  Archive 
Workshops,  held  at  ARI-POM,  as  analytical  tools. 


Task  15 


Recommend  Follow-On  Research 


Summary 

As  per  agreement  with  the  COR,  this  contract  requirement  was  satisfied  by 
providing,  in  this  final  report,  paragraphs  describing  current  and  anticipated 
utilization  of  products  for  each  contract  task. 


Product s/Reports 

Lewman,  T.  J.,  (Ed.)  (1988,  August).  Final  Report  for  Contract  No.  MDA903- 

85-C-0472  (ARI  Researh  Note).  Presidio  of  Monterey,  CA:  U.S.  Army 
Research  Institute  Field  Unit. 


Utilization 


This  report  summarizes  and  documents  three  years  of  work  in  the  areas  of 
standardization,  structuring,  and  analysis  of  NTC  data.  The  report  serves  as  a 
reference  for  ARI  personnel  and  others  who  are  working  in  these  areas. 
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